Research Paper Business Analytics Predicting participant behavior and dropout in a physical activity experiment using machine learning

Physical inactivity is identified by the World Health Organization as the fourth leading risk factor for global mortality as it increases the risk of various adverse health conditions. App-based health interventions promise to help increase physical activity levels by enhancing the motivation of the users to exercise on a regular basis. One such app-based intervention which sends tailored coaching messages to help people become more physically active, has been created by the Active2Gether project. A 12-week physical activity experiment was conducted with 92 healthy young adults to study the effect of the Active2Gether app. In this paper, supervised machine learning is applied on the experimental data, in order to detect which factors together best predict behavioral increase and which predict dropout from the experiment. Several tree-based classifiers are fitted and tuned in order to optimize F1-score. The classifiers are evaluated using K-Fold cross validation and furthermore a modified repeated cross validation is applied in order to evaluate the robustness of the models. For predicting behavioral increase, we fitted Decision Tree, Random Forest and Extremely Randomized Trees (Extra-Trees). Results show that the Decision Tree outperformed the Random Forest and the Extra-Trees, achieving an F1-score 0.8 in the 5-fold cross validation and 0.72 in the repeated cross validation. For participant dropout, we compared Decision Tree, Extra-Trees and Extreme Gradient Boosting (XGBoost). Results show that the Extra-Trees performed best with an F1-score 0.765 in the 10-fold cross validation and 0.68 in the repeated cross validation.